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(54) Method and apparatus for producing three-dimensional ultrasonic images in quasi real time 



(57) Disclosed is an apparatus and method for pro- 
ducing a three-dimensional (3D) moving image of a 
moving target object. The method comprises the steps 
of transmitting and receiving ultrasound signals to /from 
the moving target object (S202), obtaining raw data 
(S204); generating image frame data based on the ob- 
tained raw data; generating virtual image frame data 
based on the generated image frame data; and render- 
ing (S206) a 3D moving image of the moving target ob- 
ject using the image and virtual image frame data. The 
apparatus comprises a probe, a raw 3D data obtaining 
unit, and an enhanced live 3D imaging unitforproducing 
a 3D image of the moving target object. 
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Description 

* 

[0001] The present invention relates generally to ultrasound imaging, and more particularly, to a three-dimensional 
(3D) ultrasound imaging method and apparatus for producing a 3D moving image of a moving target object. 

5 [0002] Static 3D images are conventionally produced by obtaining raw 3D data, e.g., data on a coordinate system 
(x, y, z), through a 3D probe regardless of acquisition time, by stacking frames over one another at a uniform time 
interval to form consecutive frames, and by processing the consecutive frames using a 3D rendering technique. Where 
static 3D images are used for ultrasound diagnostic purposes, one may easily make accurate observation, diagnosis, 
or treatment of the internal state of a human body without performing complicated procedures associated with invasive 

10. operations. Thus, static 3D images are widely used. However, static 3D images are not useful in observing a moving 
target object in real time, such as an embryo in the uterus. 

[0003] In order to overcome this shortcoming, a live 3D imaging method and apparatus for providing a 3D moving 
image, rather than static 3D images, has been developed. Herein, a live 3D image should be understood and interpreted 
as a quasi real-time 3D moving image, representing movement of a moving target object. The live 3D image consists 

15 of fewer frames than those of a real-time 3D moving image so that it does not completely represent movement of a 
moving target object. However since the live 3D image consists of more frames than static 3D images, e.g., 2 to 4 
frames per second, it can represent movement of a moving target object more smoothly than the static 3D images. 
[0004] Referring to Fig. 1 . which illustrates a schematic block diagram of a conventional ultrasound apparatus for 
producing alive 3D image, live 3D imaging apparatus 1 00 comprises probe 1 02 for transmitting and receiving ultrasound 

20 signals to/from a moving target object (not shown), i.e., scanning the moving target object; image processing unit 104 
for producing a live 3D image by using the reflected ultrasound signals transmitted from probe 102; and display unit 
106 for displaying the live 3D image produced by image processor 104. 

[0005] Image processing unit 104 includes raw 3D data obtaining unit 108 and rendering unit 110. Raw 3D data 
obtaining unit 1 08 obtains raw data required for producing the live 3D image based on the reflected ultrasound signals 

25 transmitted from probe 102, which are transmitted to rendering unit 110. (Hereinafter, raw data refers to raw 3D data) 
Raw 3D data obtaining unit 108 may sequentially store raw 3D data corresponding to each frame on a predetermined 
storage unit (not shown), which are transmitted to rendering unit 1 1 0. Rendering unit 110 then performs a conventional 
rendering process on the raw 3D data transmitted from raw 3D data obtaining unit 1 08 and produces the live 3D image 
to be displayed on display unit 106. 

30 [0006] Referring to Fig. 2, which shows a flow chart for conventionally producing a live 3D image, probe 1 02 scans 
a moving target object by transmitting ultrasound signals to the moving target object and receiving the ultrasound 
signals reflected therefrom (Step S202). Raw 3D data obtaining unit 108 of image processing unit 1 04 obtains raw 3D 
data and stores them on a predetermined storage unit (Step S204). Where the stored raw 3D data correspond to n 
number of frames, rendering unit 110 reads raw 3D data corresponding to the (n-l)* frame among n number of the 

35 frames, which is stored just before the n th frame, and performs a high-speed rendering on the (n^) 1 * 1 frame, which is 
stored just before the (n-l)^ frame (Step S206). By performing such parallel rendering processes repetitively, rendering 
unit 110 performs live 3D imaging to produce the live 3D image of the moving target object. Display unit 106 displays 
the live 3D image thereon (Step S208). The frame rate of the live 3D image displayed on display unit 106 is typically 
2 to 4 frames/second. 

40 [0007] As described above, with conventional live 3D imaging apparatus 100, displaying images like a cinema is 
very difficult, since the live 3D image consists of 2 to 4 frames per second while the cinema typically consists of at least 
30 frames per second. For example, if a frame rate of the live 3D image is 4 frames per second, the live 3D image may 
be discontinuously displayed for one second. 

[0008] Thus, need exists for an apparatus and method for providing a natural looking 3D moving image by raising 
45 the conventional frame rate of live 3D image without increasing the calculation load of an live 3D imaging apparatus. 
[0009] Therefore, an objective of the present invention is to provide an ultrasound imaging apparatus and method 
for producing a live 3D image of a moving target object, which looks more natural than a conventional live 3D image, 
by raising the frame rate of conventional live 3D image by inserting a predetermined number of virtual image frames 
at a uniform time interval between the live 3D image frames obtained from raw 3D data of the moving target object. 
50 [0010] In accordance with one aspect of the present invention, an ultrasound imaging method for producing a 3D 
moving image of a moving target object, comprises the steps of: a) transmitting ultrasound signals to the moving target 
object and receiving reflected ultrasound signals; b) obtaining raw data based on the reflected ultrasound signals; c) 
generating image frame data including a plurality of image frames based on the obtained raw data; d) generating virtual 
image frame data including a multiplicity of virtual image frames, which are to be inserted between the plurality of image 
55 frames, based on the image frame data; and e) rendering a 3D moving image of the moving target object using the 
image frame and virtual image frame data. 

[0011] In accordance with another aspect of the present invention, an ultrasound imaging apparatus for producing 
a 3D moving image of a moving target object, comprises: a probe for transmitting ultrasound signals to the moving 
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target object and receiving reflected ultrasound signals; means for obtaining raw data based on the reflected ultrasound 

- signals; and means for generating image frame data including a plurality of image frames based on the obtained raw 
data, generating virtual image frame data including a multiplicity of virtual image frames, which are to be inserted 
between the plurality of image frames, based on the image frame data, and rendering a 3D moving image of the moving 

5 target object using the image frame and virtual image frame data. 

- [0012] The features of the present invention, which are believed to be novel, are set forth with particularity in the 
appended claims. The present invention, both as to its organization and manner of operation, together with further 
objects and advantages thereof, may best be understood with reference to the following description, taken in conjunc- 
tion with the accompanying drawings. 



Fig. 1 is a schematic block diagram of a conventional ultrasound imaging apparatus for producing a live three- 
dimensional (3D) image of a moving target object. 

Fig. 2 illustrates a flow chart for conventionally producing a live 3D image of a moving target object. 
Fig. 3 is a schematic block diagram of an ultrasound imaging apparatus for producing a live 3D image of a moving 
15 target object in accordance with the present invention. 

Fig. 4A illustrates virtual image frames produced by using interpolation in accordance with the present invention. 
Fig. 4B illustrates virtual image frames produced by using extrapolation in accordance with the present invention. 
Fig. 4C shows an example of the virtual image frames produced by using interpolation in accordance with the 
present invention. 

20 Fig. 5 illustrates a flow chart for producing a live 3D image of a moving target object in accordance with the present 

invention. 

[0013] Referring to Fig. 3, which shows a schematic block diagram of an apparatus for producing a live 3D image in 
accordance with the present invention, live 3D imaging apparatus 300 comprises probe 302 for transmitting ultrasound 

^5 signals to a moving target object (not shown) and receiving the ultrasound signals reflected therefrom, image processing 
unit 310 for producing a live 3D image of the moving target object based on the reflected ultrasound signals transmitted 
from probe 302, and display unit 308 for displaying the live 3D image produced by image processing unit 31 0. 
[0014] Image processing unit 310 includes raw 3D data obtaining unit 304 and enhanced live 3D imaging unit 306. 
Raw 3D data obtaining unit 304 of image processing unit 310 obtains raw data required for producing the live 3D image 

30 based on the reflected ultrasound signals transmitted from probe 302, which are transmitted to enhanced live 3D 
imagmg 306. Raw 3D data obtaining unit 304 sequentially transmits raw 3D data corresponding to each frame of the 
Irve 3D image to enhanced live 3D imaging unit 306. Enhanced live 3D imaging unit 306 produces data including the 
frames of the live 3D image (hereinafter, the original live 3D image frames) to be displayed on display unit 308 based 
on the raw 3D data transmitted from raw 3D data obtaining unit 304. Enhanced live 3D imaging unit 306 then produces 

35 data including virtual image frames to be inserted between the original live 3D image frames based on the original live 
3D image frame data. Thereafter, enhanced live 3D imaging unit 306 performs 3D rendering on a live 3D image in 
which the original live 3D image frames and the virtual image frames are mixed together, using the original live 3D 
image frame data and the virtual image frame data. Enhanced live 3D imaging unit 306 employs interpolation or ex- 
trapolation, to produce the virtual image frame data. 

40 [0015] Fig. 4A illustrates (n-1) th frame 402 and n th frame 406 among n number of the original live 3D image frames, 
and m number of virtual image frames 404 which are produced by using interpolation and inserted between (n-1) th 
frame 402 and n th frame 406 among the original live 3D image frames. Fig. 4B illustrates m number of virtual image 
frames 408 produced by using extrapolation. Virtual image frames 408 are added next to n lh frame 406 of the original 
live 3D image frames. Fig. 4C shows an example of virtual image frames produced by using interpolation as described 

45 with reference to Fig. 4A. As may be seen from Fig. 4C, movement of an embryo is represented naturally since the 
virtual image frames produced are inserted between the (n-1) th frame and n th frame among the original live 3D image 
frames. Herein, the total number of the original live 3D image frames is n and the total number of virtual image frames 
produced through interpolation or extrapolation is m, wherein n and m are integers. 

[0016] As illustrated in Fig. 4A : where enhanced live 3D imaging unit 306 employs linear interpolation, display time 
so t a j of the j th virtual image frame among m number of virtual image frames 404 and a color value y a j corresponding to 
one of pixels in the j th virtual image frame may be defined as follows: 
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TYl . 

5 . .v (Eq. 1) 

(m—j) j 

*o. wherein y aj is a color value corresponding to a pixel having a coordinate (t aj , y aj ) among the pixels in the virtual 
image frame; y n is a color value corresponding to a predetermined pixel in one of the original live 3D image frames, 
which is displayed at time t„; m is the total number of virtual image frames 404 inserted between (n-1 ) th frame 402 and 
n th frame 406; and a is a pixel among the pixels in one of virtual image frames 404. Data including m number of virtual 
image frames 404 are obtained by applying Equation 1 to every pixel in each of virtual image frames 404. (Hereinafter, 

is virtual image frame data refers to the data including m number of virtual image frames 404) Similarly, when receiving 
data including the (n+1)^ frame (not shown) among the original live 3D image frames, enhance live 3D imaging unit 
306 generates virtual image frame data including virtual image frames (not shown) to be inserted between n th frame 
406 and (n+1) th frame by repeating the above. 

[0017] Where enhanced live 3D imaging unit 306 employs high-order interpolation instead of linear interpolation, the 
20 n* 1 polynomial expression of interpolation may be expressed as follows: 



y= c 0 + a,f + c 2 ? +...+ c n f (Eq. 2) 



wherein c 0 to c n arc arbitrary coefficients; t is the display time of the live 3D image; yis virtual image frame data; and, 
n is the total number of the original live 3D image frames. 
[0018] Where m number of virtual image frames to be inserted between (n-1 )th frame 402 and n th frame 406 among 
the original live 3D image frames are generated, display time t aj of the j th virtual image frame among m number of the 
virtual image frames generated may be expressed by Equation 1 as described above. At display time t aj) a color value 
30 corresponding to a predetermined pixel in the j 1h virtual image frame may be expressed as follows: 

Yaj = c 0 + c 1 W + c 2 ? aj +•"+ c n f1 aj (Eq. 3). 



[0019] Data including the j th virtual image frame are obtained by calculating a color value corresponding to every 
pixel in the j th virtual image frame among m number of the virtual image frames through Equation 3. Where the number 
of the original live 3D image frames is F, the total number of image frames to be displayed per second, which is obtained 
by using the linear interpolation and high-order interpolation described above, becomes F'On+l). 
[0020] As illustrated in Fig. 4B, where enhanced live 3D imaging unit 306 employs high-order extrapolation, display 
time t ej of the j th virtual image frame among m number of virtual image frames 408 and a color value y ej corresponding 
to one of pixels in the j th virtual image frame are defined as follows: 



45 t ej > t n 

(Eq-4) 

y eJ = c 0 + c x t cj + c 2 t\j + . . - + c n t% 



wherein y ej is a color value corresponding to a pixel having a coordinate (t ej , y ej ) among the pixels in the j th virtual 
image frame. 

[0021] Data including the j th virtual image frame are obtained by calculating a color value corresponding to every 
pixel in the j th virtual image frame among m number of the virtual image frames through Equation 4. Where the number 
of the original live 3D image frames is F, the total number of image frames to be displayed per second, which is obtained 
by using high-order extrapolation described above, becomes F*(m+1). 

[0022] Although enhanced live 3D imaging unit 306 has been described to produce the virtual image frames to be 
inserted between the original live 3D image frames by using linear interpolation, high-order interpolation, and high- 
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order extrapolation, enhanced live 3D imaging unit 306 may produce the virtual image frames by using any appropriate 

• technique. 

[0023] The frame rate of a live 3D image may vary with conditions such as the kind of probe, depth of ultrasound 
images, and acquisition rate of 3D images. By controlling the number of virtual image frames obtained through inter- 
5 polation or extrapolation, the operation time for interpolation or extrapolation should be minimized and the virtual image 

• frames should be inserted and displayed at a uniform time interval between the original live 3D image frames. For 
example, where the frame rate of original live 3D image frames is Ft lames/second, data including Fframes per second 
are obtained. Accordingly, where m number of virtual image frames are to be inserted between the original live 3D 
image frames, the time for performing interpolation or extrapolation on m number of the virtual image frames must not 

10 . exceed 1/(F(n?+1)) seconds, since a live 3D image containing the original live 3D image frames and the inserted virtual 
image frames should be displayed at every 1/(F(m+1)) second. Therefore, determining the number of virtual image 
frames, which may be obtained within 1/(F(#n+1)) seconds, is necessary. 

[0024] Referring to Fig. 5, which illustrates a flow chart for producing a live 3D image in accordance with the present 
invention, probe 302 scans a moving target object by transmitting ultrasound signals to the moving target object and 

15 receiving the ultrasound signals reflected therefrom (Step S502). Raw 3D data obtaining unit 304 of image processing 
unit 310 obtains and stores raw 3D data based on the reflected ultrasound signals transmitted from probe 302 (Step 
S504). Enhanced live 3D imaging unit 306 generates data including original live 3D image frames based on the raw 
3D data obtained by raw 3D data obtaining unit 304 and produces data including virtual image frames to be inserted 
between the original live 3D image frames (Step S506). Enhanced live 3D imaging 306 then renders a live 3D image 

20 of the moving target object using the original 3D image data and the virtual image data (Step S508). Display unit 308 
displays thereon the live 3D image, i.e., a 3D moving image (Step S510). 

[0025] As described above, inserting the virtual image Frames between the original live 3D image frames raises the 
frame rate of the live 3D image being displayed. Thus, a more-natural-looking, live 3D image, i.e., a 3D moving image, 
compared to the conventional live 3D image, is obtained without increasing the calculation load of a live 3D imaging 
25 apparatus. 

[0026] While the present invention has been shown and described with respect to the particular embodiments, those 
skilled in the art will recognize that many changes and modifications may be made without departing from the scope 
of the invention as defined in the appended claims. 

30 

Claims 

1 . A method for producing a three-dimensional (3D) moving image of a moving target object, comprising the steps of: 
35 a) transmitting ultrasound signals to the moving target object and receiving reflected ultrasound signals; 

b) obtaining raw data based on the reflected ultrasound signals ; 

c) generating image frame data including a plurality of image frames based on the obtained raw data; 

40 

d) generating virtual image frame data including a multiplicity of virtual image frames, which are to be inserted 
between the plurality of image frames, based on the image frame data; and 

e) rendering a 3D moving image of the moving target object using the image frame and virtual image frame data. 

45 

2. The method of Claim 1 , wherein the virtual image frame data are generated by using one of interpolation or ex- 
trapolation. 

3. The method of Claim 2, wherein the multiplicity of virtual image frames are obtained within 1/(/=*(m+1)) seconds, 
50 wherein a frame rate of the plurality of image frames is Fframes / second and m is the total number of the virtual 

image frames. 

4. An apparatus for producing a three-dimensional (3D) moving image of a moving target object, comprising: 

55 a probe (1 02,302) for transmitting ultrasound signals to the moving target object and receiving reflected ultra- 

sound signals; 

means (108,304) for obtaining raw data based on the reflected ultrasound signals; and means for generating 
image frame data including a plurality of image frames based on the obtained raw data, generating virtual 
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image frame data including a multiplicity of virtual image frames, which are to be inserted between the plurality 
of image frames, based on the image frame data, and rendering a 3D moving image of the moving target 
object using the image frame and virtual image frame data. 

5 5. The apparatus of Claim 4, wherein the virtual image frame data are generated by using one of interpolation or 
extrapolation. 

6. The apparatus of Claim 5, wherein the multiplicity of virtual image frames are obtained within 1 /(P(/ih-1 )) seconds, 
wherein a frame rate of the plurality of image frames is F frames / second and m is the total number of the virtual 
10. image frames. 
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Fig. 2 
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